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LONG-TERM  GOALS 

To  predict  the  probability  distribution  function  (pdf)  of  medium  range  weather  forecast  errors  as 
accurately  as  possible. 

OBJECTIVES 

Objective  1:  To  compare  the  Bishop  et  al.’s  (2001)  recently  developed  Ensemble  Transform  Kalman 
Filter  (ET  KF)  ensemble  generation  technique  against  the  breeding  of  growing  vectors  (BGV)  technique 
(Toth  and  Kalnay,  1993,  1997)  in  a  GCM. 

Objective  2:  To  compare  simplex  and  symmetric  ensemble  centering  techniques 

Objective  3:  To  develop  statistical  “dressing”  techniques  that  ensure  that  the  difference  between  any 
two  members  of  an  ensemble  is  statistically  indistinguishable  from  the  difference  between  any  one 
ensemble  member  and  the  truth. 

APPROACH 

Users  of  weather  forecasts  with  non-linear  cost/loss  functions  can  benefit  from  details  of  the  probability 
distribution  function  (pdf)  of  future  atmospheric  states  given  past  and  current  observations,  a  data 
assimilation  scheme  and  forecast  models.  The  above  objectives  represent  sub-goals  aimed  at  the  over¬ 
arching  aim  of  providing  users  with  detailed  and  reliable  probabilistic  forecasts  of  future  atmospheric 
states.  Our  test-bed  is  NCAR’s  community  climatemodel  (CCM3)  run  off  NCAR/NCEP  reanalysis 
data. 


WORK  COMPLETED 


Considerable  progress  has  already  been  made  in  all  of  the  above  objectives.  Xuguang  Wang,  the  Penn 
State  PhD  student  who  is  paid  by  this  ONR  grant,  has  developed  numerical  tools  to  run  and  test  T42 
CCM3  ensemble  forecasts  off  the  NCEP/NCAR  reanalysis  data  set  on  8  PCs  each  with  dual  1  GHz 
processes  running  Linux.  As  a  result  of  this  work,  one  paper  (Wang  and  Bishop,  2003)  entitled  “A 
comparison  of  Breeding  and  Ensemble  Transform  Kalman  Filter  Forecast  Schemes”  appeared  in  the 
Journal  of  Atmospheric  Sciences  earlier  this  year  while  another  (Wang  et  al.,  2003)  entitled  “Which  is 
better,  an  ensemble  of  positive/negative  pairs  or  a  centered  spherical  simplex  ensemble?”  has  been 
submitted  for  publication  in  the  Monthly  Weather  Review.  A  third  paper  is  currently  being  prepared  for 
submission  to  the  Quarterly  Journal  of  the  Royal  Meteorological  Society  on  the  subject  of  statistical 
“dressing”  techniques  that  ensure  that  the  difference  between  any  two  members  of  an  ensemble  is 
statistically  indistinguishable  from  the  difference  between  any  one  ensemble  member  and  the  truth. 

MAIN  RESULTS 

See  Wang  and  Bishop  (2003)  for  results  on  “A  comparison  of  breeding  and  ensemble  transform 
Kalman  filter  ensemble  forecast  schemes”.  Results  for  the  “Which  is  better,  an  ensemble  of 
positive/negative  pairs  or  a  centered  spherical  simplex  ensemble?”  can  be  found  at 
http://www.essc.psu.edu/~xuguang/publications.html.  Here  we  focus  on  our  recent  results 
regarding  the  statistical  augmentation  of  ensemble  perturbations. 

Statistical  post-processing  with  the  dressing  technique 

To  account  residual  errors,  one  way  we  can  try  is  to  add  statistical  perturbations  to  each  of  the 
dynamical  ensemble  members  in  the  post- processing.  This  idea  is  first  tried  by  the  best  member 
dressing  technique  by  Roulston  and  Smith  (2003).  In  the  best  member  dressing  method,  the  statistical 
perturbations  are  from  archived  historical  best  member  errors.  The  best  member  is  defined  as  the 
closest  to  the  verification  in  the  full  space  including  all  spatial  locations,  all  quantities  and  all  forecast  lead 
times.  Our  first  concern  about  this  best  member  dressing  method  is  that  since  the  error  statistics  of  each 
member  should  be  similar  on  average  one  should  not  expect  the  first,  the  second  and  the  even  the  worst 
member  to  be  significantly  different  measured  in  full  space.  Secondly,  since  identification  of  the  best 
member  is  time  consuming,  one  would  choose  a  subspace.  Then  the  selection  of  the  best  member  is 
dependent  on  the  choice  of  subspace.  Third,  Roulston  and  Smith  (2003)  shows  that  using  too  low 
dimensional  space  will  very  likely  misidentify  the  true  best  member  defined  in  full  space  and  it  will 
underestimate  the  errors  associated  with  each  forecast.  But  is  it  possible  that  the  selection  of  the  best 
member  in  very  high  dimensional  space  will  overestimate  the  errors? 

With  these  questions  in  mind  we  propose  another  dressing  kernel.  The  basic  idea  is  to  choose  the 
statistical  perturbations  that  will  make  the  dressed  ensemble  members  indistinguishable  from  the 
verifications  under  the  second  moment  measurements.  Mathematically, 


where  y  Jp  and  y  £  are  two  randomly  picked  dressed  ensemble  members, 
the  seasonal  average.  Then  the  dressing  perturbation  e  satisfies 


xr  is  the  verification.  (  )  is 


where  ?  is  the  undressed  ensemble  mean  and  S(2  is  the  seasonally  averaged  undressed  ensemble 


covariance. 

Our  proposed  method  has  shown  better  results  relative  to  the  best  member  method  when  we 
test  with  single  variable  of  interest  (see  fig. 16,  17  and  18).  We  have  developed  a  mathematical  tool  to 
extend  the  dressing  to  more  variables,  in  which  case  the  covariance  between  different  elements  need  to 
be  considered.  Work  is  underway  to  test  this  tool. 
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Fig.  16  Test  of  the  two  dressing  technique.  RS:  best  member  method.  WB:  proposed  method.  The  undressed 
ensemble  is  16  member  ETKF  ensemble.  2  dressing  perturbations  are  added  to  each  of  the  16  ETKF  ensemble 
members.  The  single  variable  considered  are  500mb  U  at  Eastern  USA.  In  the  best  member  method,  the  best  member 
is  defined  from  a  very  high  dimensional  subspace:  200mb,  500mb  and  850mb  U,V,  T  throughout  1  to  10  day  forecast 
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Fig.  17  Brier  skill  score  measurements  on  the  three  ensembles  in  fig.  16.  Our  proposed  method  shows  smaller  BSS. 
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Fig.  18  Contiuouse  ranked  probablility  score  measurements  on  the  three  ensembles  in  fig.  16.  Ourproposed  method 
shows  smaller  CRPS. 


IMPACT/APPLICATIONS 

At  NRL  Monterey,  research  is  being  conducted  to  improve  FNMOC’s  (bred  vector)  ensemble 
forecasting  capabilities.  Because  of  the  positive  results  found  in  our  prehminary  tests,  the  ETKF 
ensemble  generation  scheme  and  other  schemes  will  be  tested  at  NRL  to  determine  their  suitability  for 
transition  into  operations  at  FMNOC.  Zoltan  Toth  and  Mozheng  Wei  of  the  National  Centers  for 
Environmental  Prediction  (NCEP)  are  also  preparing  to  test  versions  of  the  ETKF  ensemble  generation 
scheme  to  determine  its  suitability  as  a  replacement  to  their  current  bred  vector  scheme. 

TRANSITIONS 

NCEP,  in  collaboration  with  former  Post-doctoral  fellow  Sharanya  Majumdar,  graduate  student  Brian 
J.  Etherton  and  undergraduate  student  Jonathon  Moskaitis,  is  currently  applying  the  ETKF  to  a 
combined  ECMWF/NCEP  ensemble  to  determine  were  aircraft  should  fly  in  the  ongoing  NOAA 
Winter  Storms  Reconnaissance  program. 

RELATED  PROJECTS 

The  NSF  grant  ATM-98- 14376  “Adaptive  Sampling  with  the  Ensemble  Transform  Kalman  filter” 
enabled  tests  of  the  ability  of  the  ETKF  to  predict  reductions  in  forecast  error  variance  due  to  targeted 


observations.  See  http://www.met.psu.edu/dept/faculty/bishop.htmand 
http://orca.rsmas.miami.edu/~majumdaiy  for  details. 

SUMMARY 

In  order  to  more  accurately  represent  the  uncertainty  in  weather  forecasts  a  new,  computationally 
inexpensive  method  has  been  devised  for  generating  multiple  forecasts  whose  differences  reflect  weather 
forecast  uncertainty.  Our  tests  indicate  that  the  method  is  superior  to  the  breeding  technique  that  is 
currently  used  by  the  federally  funded  civilian  and  Naval  weather  forecasting  agencies.  In  addition  new 
ensemble  statistical  techniques  have  been  introduced  that  augment  synamical  ensemble  members  with 
statistical  members.  These  dressed  ensembles  are  designed  so  that  the  difference  between  any  two 
dressed  ensemble  members  is  statistically  indistinguishable  from  the  difference  between  any  one 
ensemble  member  and  the  verifying  observation.  Thus,  the  statistical  augmentation  technique  greatly 
enhances  the  value  of  the  ensemble  forecast  to  users  with  non-linear  cost  functionals. 
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